Video compression coders and/or decoders (CODECs) gain much of their compression from interframe prediction. However, the simple interframe technique cannot sufficiently improve coding efficiency when temporal brightness variation is involved.
The H.264/JVT/MPEG AVC (“H.264”) video compression standard provides a weighted prediction tool. This works well for global brightness variation, but due to the limitation of the number of different weighting parameters that can be used, little gain can be achieved in the presence of significant local brightness variation.
The interframe prediction process forms a reference picture prediction of a picture to be encoded, and encodes the difference between the current picture and the prediction. The more closely correlated the prediction is to the current picture, the fewer the bits that are needed to compress the picture.
In prior video CODECs, the reference picture is formed using a previously decoded picture. Unfortunately, when serious temporal brightness variation is involved, e.g. due to illumination changes, fade-in/out effects, camera flashes, etc., conventional motion compensation can fail (or become extremely inefficient).
In H.264, a weighted prediction (WP) tool is used to improve coding efficiency. WP estimates the brightness variation by a multiplicative weighting factor a and an additive weighting offset b as in equation (eq: 1).I(x,y,t)=a·I(x+mvx,y+mvy,t−1)+b  (1)where I(x,y,t) is the brightness intensity of pixel (x,y) at time t, a and b are constant values in the measurement region, and (mvx,mvy) is the motion vector.
Weighted prediction is supported in the Main and Extended profiles of the H.264 standard. The use of weighted prediction is indicated in the picture parameter sets for P and SP slices using the weighted_pred_flag field, and for B slices using the weighted_bipred_idc field. There are two WP modes, an explicit mode and an implicit mode. The explicit mode is supported in P, SP, and B slices. The implicit mode is supported in B slices only.
In WP, the weighting factor used is based on the reference picture index (or indices in the case of bi-prediction) of the current macroblock or macroblock partition. The reference picture indices are either coded in the bitstream or may be derived, e.g., for skipped or direct mode macroblocks. A single weighting factor and a single offset are associated with each reference picture index for all slices of the current picture. For the explicit mode, these parameters are coded in the slice header. For the implicit mode, these parameters are derived. The weighting factors and offset parameter values are also constrained to allow 16 bit arithmetic operations in the inter prediction process.
The explicit mode is indicated by weighted_pred_flag equal to 1 in P or SP slices, or by weighted_bipred_idc equal to 1 in B slices. As previously stated, in this mode, the WP parameters are coded in the slice header. A multiplicative weighting factor and an additive offset for each color component may be coded for each of the allowable reference pictures in list 0 for P slices and B slices. The number of allowable reference pictures in list 0 is indicated by num_ref_idx_I0_active_minus1, while for list 1 (for B slices) this is indicated by num_ref_idx_I1_active_minus1.
For global brightness variation that is uniformly applied across an entire picture, a single weighting factor and offset are sufficient to efficiently code all macroblocks in a picture that are predicted from the same reference picture. However, for brightness variation that is non-uniformly applied, e.g. for lighting changes or camera flashes, more than one reference picture index can be associated with a particular reference picture store by using reference picture reordering. This allows different macroblocks in the same picture to use different weighting factors even when predicted from the same reference picture. Nevertheless, the number of reference pictures that can be used in H.264 is restricted by the current level and profile, or is constrained by the complexity of motion estimation. This can considerably limit the efficiency of WP during local brightness variations.
Accordingly, it would be desirable and highly advantageous to have a method and apparatus for weighted prediction video coding that overcomes at least the above-identified deficiencies of the prior art.